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1. Introduction 


Today’s use of technology and media in daily life has altered the dominant 
role the written word has played in communication over centuries. Currently, 
educational settings take into consideration the combination of different modes 
of representation that exist in an array of everyday texts. “Developing knowledge 
about linguistic, visual and digital meaning-making systems” (Unsworth, 2001, 
p. 10) has become a key learning objective. In the digital era, students develop 
literacy by understanding the organization and display of information through 
multiple modes of communication and the ways these different modes cooperate 
in the creation of meanings (Jewitt, 2009; Kress, 2009; Lemke, 1998; Unsworth, 
2001). As a result, teaching through and about media is crucial in foreign 
language classrooms to promote transcultural and translingual competencies 
(Lebrun, Lacelle, & Boutin, 2012). Yet foreign language educators must learn 
how the relationships (Kern, 2006) or orchestration (Kress & van Leeuwen, 
1996; Nelson, 2006) within and across semiotic modes (Kress, 2003), including 
language, facilitate the ways learners create meaning. Developing activities that 
teach students the tools of multimodal texts and how to reapply these resources 
in a personal way forces teachers to reflect upon ways their teaching practices 
facilitate language learning (Anderson, Chung, & Macleroy, 2018; Jiang, 2017; 
Ollivier, 2018). 
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Chapter 5 


Conducted in an intermediate French-language curriculum grounded in 
the pedagogy of multiliteracies (New London Group, 1996), and Cope and 
Kalantzis’s (2015) framework of learning by design, this study investigates 
Digital Storytelling (DS) (Lambert, 2002) within a social synesthetic semiosis 
(Nelson, 2006; Oskoz & Elola, 2016; Yang, 2012) with particular attention to 
the transformation and transduction processes (Bezemer & Jewitt, 2009; Kress, 
2003). In other words, I explore how DS supports students’ selection and 
orchestration of semiotics to construct layers of meaning and foster language 
development. 


Findings from two case studies of fourth-semester French learners highlight 
participants’ ability to circumvent challenges and convey their stories in 
multiple modes, including the target language. The study underscores to foreign 
language teachers the potential of media projects anchored in the multiliteracies 
framework to enhance students’ media literacy skills as they critically reflect on 
the use of media from the perspective of both consumers and producers. 


2. Background 
2.1. Multiliteracies 


These past decades, within the field of literacy studies, numerous educators 
and scholars (Cope & Kalantzis, 2009, 2015; Gee, 2008; Kern, 2000, 2015; 
Kress, 2003; New London Group, 1996) have stressed the changes occurring 
as a result of new social practices and discussed what it means to be literate in 
today’s world. The concept of being literate has shifted from knowing how to 
read and write printed text to gaining the ability to read and produce varied texts 
across a set of social and cultural contexts through multiple digital devices. 
Today’s texts are produced, distributed, and consumed through visual, aural, 
sensorial, spatial, and gestural modes which, when combined, communicate 
particular meanings, achieve specific purposes, and reach certain audiences. 
Thus, language — in its linguistic dimension — can no longer be considered 
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the sole mode of conveying messages. Foreign language educators should no 
longer privilege the linguistic mode at the expense of other modes. Instead, 
they should address these new literacies and new ways of creating meanings to 
help students navigate through and negotiate with multimodal texts and their 
meanings to become multiliterate. 


According to Lebrun etal. (2012), the teaching of new literacies and multiliteracies 
is central in foreign language classrooms, and a few studies have demonstrated 
the significance of its implementation. Most instructors teach using multimodal 
texts rather than teaching how to read and write them. Because instructors lack 
knowledge of what makes a text multimodal, they privilege the linguistic mode 
over other semiotic resources. Thus, scholars promote the integration of teaching 
new literacies into language curricula and stress the importance of incorporating 
innovative pedagogical frameworks into teachers’ professional development 
(Anderson et al., 2018; Ollivier, 2018; Oskoz & Elola, 2016). 


Although the implementation of these frameworks is challenging and time- 
consuming, they allow teachers to reflect on their teaching practices. The concept 
of design (Cope & Kalantzis, 2009, 2015; Kern, 2015) in the multiliteracies 
framework proposes developing a classroom application as an active process 
of transformation from the known to the new. Through acts of designing and 
redesigning, students build knowledge. Design elements included in text, image, 
sound, gesture, space, and sense allow students to move back and forth among 
the modes of representation and foster meaning-making. Such multimodal design 
provides interconnection across and between other modes, and learning emerges 
from mode switching. Students become more sensitive to the semiotics used 
in texts and their meaning potential. They gain the ability to make intentional 
choices while producing their own multimodal texts through a variety of means 
in the target language (Jiang, 2017). 


Furthermore, the learning by design approach applies to both learners and 
teachers. The former take control of their apprenticeship to become designers of 
their knowledge through action-taking, collaboration, and active participation in 
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and outside formal learning (Anderson et al., 2018). The latter plan and organize 
teaching sequences around multimodal texts, identify learning strategies, and 
reflect on assessment and learning outcomes. 


Moreover, instructors should experiment with innovative and collaborative 
approaches to fulfilling foreign language learners’ needs and fostering learning 
competencies as a whole rather than in isolation (Anderson et al., 2018). Across 
scholarships, there emerge various implementations of experiential learning 
in foreign language education. These curricula have provided L2 learning and 
teaching approaches that fostered multiliteracies, such as the use of social media 
(Reinhardt & Zander, 2011), gaming (Reinhardt, Warner, & Lange, 2014), social 
reading (Blyth, 2014), and global simulation (Michelson & Dupuy, 2014). 


2.2. DS and language learning 


DS is a textual narrative embedded with other modes of communication 
(Alismail, 2015; Robin, 2006). It consists of “short, two to three-minute mini- 
films usually based on still photos brought into a multimedia format, with a 
textual narrative read with the narrator’s voice” (Lundby, 2008, p. 366). The 
multimodal dimension of a DS empowers digital storytellers who engage with 
multiple modes of representation that have an “exponentially more complex 
impact” on themselves and their audience (Lambert, 2013). 


Many studies have examined the integration of digital stories in educational 
settings, but few studies have explored DS in Foreign Language (FL/L2) 
learning contexts. Studies have shown that DS, through formal and informal 
learning, have a beneficial impact on students’ cognition and language learning, 
as well as on their technology, media, and social competencies (Anderson et 
al., 2018; Burgess, 2006; Podkalicka & Campbell, 2010; Vinogradova, Linville, 
& Bickel, 2011). In developing their digital stories, students learn how to 
collect information, using technology, or search in the ‘real’ world (e.g. taking 
pictures, composing music or recording sounds, and interviewing members 
of a community). Through their production, students combine various modes 
(soundtrack, voiceovers, and images) and genres (interviews, documentaries, 
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and moving and still images). They develop coherent narratives in which they 
can express their emotions and values. In addition, they gain the ability to 
compose stories using technology and to collect and arrange textual, visual, and 
audio elements, as well as to perform orally (Anderson et al., 2018; Burgess, 
2006; Jiang, 2017; Vinogradova et al., 2011). 


Furthermore, the implementation of DS in FL/L2 classrooms involves 
using all language modalities: writing, reading, listening, and speaking. As 
they develop the composition of their story, language learners must make 
selective linguistic choices in terms of genre conventions, morpho-syntactic 
features (e.g. tenses, vocabulary, grammar), language register, and discourse 
appropriateness. As producers of digital stories, students need to learn how to 
write narratives following genre conventions and often with a limited number 
of words. According to Paulus (1999), the use of a multiple-draft approach is 
best for practicing writing in FL/L2 classrooms. Digital stories also support 
students’ improvement in their speaking skills (Kim, 2014; Nelson, 2006). 
To perform the task of speaking, students have to practice pronunciation and 
work on their intonation. Studies on the use of online recording programs and 
self-assessment demonstrate significant improvement in speaking accuracy 
and communicative performance (Jiang, 2017; Lynch, 2007; Volle, 2005). In 
addition, the multimodal dimension of DS places students beyond the single act 
of learning how to read, write, and speak in an FL. It engages learners in viewing 
and showing, communicating through sounds and visuals, and combining 
resources to create meanings. In the same way that they make written or spoken 
choices, they must make choices around design elements including color, font, 
layouts, background, and transition effects. They have to envision their project, 
anticipate the audience’s reaction, and manage the challenges presented by the 
creation of multimodal texts (Anderson et al., 2018; Castaneda, 2013; Jiang, 
2017; Kern, 2006; Miller, 2009; Van Gils, 2005). DS allows students to learn 
how language “as one important dimension of semiosis among others” (Nelson 
& Kern, 2012, p. 61) is anchored in sociocultural contexts and interconnected 
with other modes of representation to produce meanings. It goes beyond the 
sole learning of lexico-grammatical features (e.g. syntactic structures, grammar 
rules, vocabulary lists). 
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3. Methodology 
3.1. Course context and description 


The study was conducted in two sections of a fourth-semester French course in 
which I implemented critical media literacy frameworks combined with Cope 
and Kalantzis’s (2015) knowledge processes to foster language learning and 
emergent literacies. Throughout the semester, students explored topics through 
authentic texts culturally embedded in French society. For instance, students 
designed political cartoons addressing global issues after reflecting upon the 
controversial French magazine Charlie Hebdo and the role of cartoonists 
in the world. They also made promotional posters to advertise the National 
Museum of Immigration in Paris. They created informative posters to promote 
web safety among youth and their families. Finally, they produced their 
digital story. These projects occurred throughout the semester and allowed 
students to explore multiple media representations and various linguistic and 
semiotic forms of multimodal texts. Students engaged with these texts from 
the perspective of consumers and language learners and produced media 
artifacts drawing upon the concepts of ethics, audience, and ideology of the 
target culture. 


Inspired by the Story Center’s movement and mission to “create spaces for 
listening to and sharing stories” and to provide “skills and tools that support self- 
expression, creative practice and community building” (https://www.storycenter. 
org/about), I used the Center Story’s steps into the curriculum to help students 
to develop their final project. Although tied to specific codes and conventions, 
DS offers creative writing and production, allowing students to apply and reflect 
on what they have learned during a course. Before creating their digital story, 
students explored one particular digital story that I carefully selected on the 
Story Center’s website and discussed the relationships of the semiotics chosen 
by the author. Then, as homework, students chose two different digital stories 
and reflected on what they watched and the semiotic element(s) that captured 
their attention and could potentially be used in their project. I evaluated students’ 
work at every step of the process, including French to respond to language 
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obstacles and learners’ needs. I gave feedback on the written scripts and the 
pronunciation, intonation, and language flow of the voiceover. Before the final 
submission, students were able to verify and evaluate their projects according to 
a rubric used to assess the final version of the project. 


3.2. Research questions 


This article seeks to answer the following research question: in what ways does 
DS contribute to a student’s understanding of how the selection and orchestration 
of semiotics constructs layers of meaning and impacts multiliteracy skills and 
language development? 


3.3. Participants 


I focused on two case studies of students who identified themselves as L2 
French learners and were enrolled in an intermediate French class, the final 
course of the basic language sequence at the university. Criteria for selecting 
these students included their commitment to the class, their motivation in 
learning the target language and culture, and their high level of participation in 
class. By selecting these two case studies, I intended to present a contrastive 
sample of how students in the class had chosen to develop their digital story 
with specific semiotics in mind. These examples aim to illustrate possible 
learning strategies that students, consciously or unconsciously, implemented 
to achieve their projects. 


Born in Mexico, Maize (pseudonym) arrived in the United States with her 
family when she was seven years old. At the time of the study, she was 19 
years old, a sophomore majoring in psychology and minoring in French. In 
a postsemester interview, Maize tied her interest in French to her belief that 
learning a language expands creativity and cultural knowledge. As a native 
speaker of Spanish, she enjoyed comparing the differences of the languages. 
Raised within American culture with Hispanic traditions, she was often able 
to grasp the similarities between the French and the Mexican cultures and see 
how they differed from American culture. 
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Maize’s DS is entitled “Choses Oubli¢es [Forgotten Things]” and is intended 
to raise “a global controversy” that she described as follows. “Likes, hearts, 
followers and subscribers. Every day, these things fill up our thoughts; they 
intoxicate our beliefs and influence us to behave in a way that the rest of society 
expects and wants us to behave”. 


Fanny (pseudonym) is a 20-year-old American white woman who grew up in 
Houston, Texas. Her mother was American, and her father immigrated from 
England to the United States when he was 18 years old. Fanny was an only child 
who was born with a missing arm. Her parents raised her according to the belief 
that having one arm is incidental and should not be used as an excuse for not 
doing what others can. Fanny demonstrated a strong personality, always speaking 
for herself and standing up for her opinions. At the time of the study, Fanny was 
a freshman majoring in geology and physics in the College of Sciences. Before 
undertaking the French course, Fanny spent her last high school semester in 
France at a private bilingual international school, in Paris. Although disabled, 
Fanny was not registered at the university’s Disability Resource Center and did 
not ask for any accommodations. 


Fanny’s DS is entitled “Ma vie”. In her initial proposal, the tentative title of her 
story was “The One Arm Wonder”, which she described as follows. “In 199[...], 
on August [...], I was born without my right arm. Being born this way has given 
me the beautiful opportunity to see life th[r]ough a unique perspective that has 
made me who I am today. I have learned valuable lessons that I wish to share 
with the world with the goal of teaching about perspective and the power of the 
individual”. 


For this study, I refer to data collected from these two stories as ways to best 
exemplify how DS impacts meaning-making, multiliteracies, and language 
development. In comparing and contrasting these two stories, disparities 
emerged in how the students selected and orchestrated the semiotic resources at 
their disposal. For instance, whereas Fanny, who felt that her technological skills 
were limited, presented her project as a PowerPoint slideshow, Maize created a 
complex project that incorporated photos, videos, music, transitions, and effects. 
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Another difference between Fanny and Maize lies in their choices of resources to 
tell their story. Fanny decided to tell her story by relying primarily on linguistic 
resources. In contrast, Maize developed her project around the use of visuals and 
aesthetics, avoiding the overuse of linguistic features. 


Furthermore, even though these case studies were selected as examples of 
students’ processes of design, gaps exist in the data collection as a result of 
what participants gave access to in their consent forms. While Maize agreed to 
provide a postsemester interview, Fanny did not. In addition, although consent 
forms were collected by a third party and given to me after the official release of 
final grades, participants were fully aware of the research agenda and knew that 
their work could be shown in an academic context. Thus, conflicts of interest 
may appear in a study where participants complete a classroom project that calls 
for personal statements. 


Finally, in terms of audience, the students did not display their artifacts on a 
participative website. The tasks did not bring students beyond the educational 
boundaries and therefore, although fostering digital literacies, did not “involve 
real-world processes of language use” (Ollivier, 2018, p. 36). 


The following analysis should be considered alongside these limitations. 
3.4. Data collection 


The data were collected from diverse sources: the DS steps described below and 
the students’ final artifacts. In addition, Maize’s two postsemester interviews 
were analyzed. 


Students developed their projects following six steps and used L1 except for 
Steps 4 and 5. Feedback was given for Steps 1, 3, 4, and 5. 

1. Propose a story. 

2. Complete a prequestionnaire to reflect on the design process. 

3. Develop a storyboard. 

4. Draft a narration. 
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5. Record a voiceover. 
6. After submission, complete a postquestionnaire to reflect on learning 
outcomes. 


3.5. Postsemester interviews 


Maize consented to give a postsemester interview in English. Her first interview 
was exceptionally long (44:39 min), and she agreed to give a second interview. 
The two interviews, conducted in person and recorded, took place in a university 
library study room. The first interview happened during the spring semester 
following the course, and the second occurred during the summer. The first 
interview was semistructured with direct questions about the critical media 
literacy framework and the creation of media artifacts. Less directive, the second 
interview focused on the artifact itself and the decisions made by the participant 
while designing it. 


3.6. Data analysis 


The study followed Nelson (2006) and Bezemer and Jewitt’s (2009) analytical 
focus. The analysis identifies the modes of communication (e.g. verbal, visual, 
aural, gestural, spatial) and investigates the decisions made by the participants 
to construct meanings. 


The qualitative data analysis of participants’ artifacts involved two phases. 
3.6.1. Phase 1: analytic memos and initial coding 


Based on grounded theory (Bryant & Charmaz, 2007), the analysis of data 
started with written analytic memos followed by open coding. First, I recorded 
and reported the general patterns, categories, and subcategories of the data. 
Then, I used in vivo coding for participants’ pre and postquestionnaires and 
Maize’s semistructured interviews. Interviews were transcribed in written form, 
and data were analyzed for new insights about the process of creation based on 
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the participants’ goals. Finally, I compared storyboards, written drafts, and the 
final project version to investigate any significant differences. 


3.6.2. Phase 2: multimodal arrangement 


Scrutinizing the final artifacts and the arrangement of modes helped me to 
better understand participants’ process of design while developing their 
digital stories. First, I quantitatively tabulated and coded the diverse modes of 
communication — textual, oral, aural, and visual — based on the purposes for 
using these modes. Then, I explored whether a specific mode was predominant 
or if the participants used modes evenly. Finally, I analyzed the disparities that 
emerged between and within the modes of representation. 


4. Findings 


The following section discusses how the two participants decided to select and 
use particular semiotics over others to convey meanings. Participants showed 
autonomy and personal learning strategies. 


4.1. Textual mode 


Fanny primarily concentrated her DS on her writing and decided to add captions 
to her PowerPoint slides to reach both the L2 and L1 audience. She demonstrated 
knowledge of narratology and used writing strategies to communicate her story. 
She had higher L2 proficiency than her peers and was committed to mastering 
the writing portions of her digital story by submitting multiple drafts (4) to 
receive as much feedback as possible. 


Conversely, Maize made little use of the textual mode in her DS. She only used 
it to state credits and acknowledgments, which were displayed at the end of the 
project. She did not want to use English captions to reach a broader audience and 
advocated in her interview: 
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“T think my message would reach more people if I also used the English 
language, but I also think sticking to the French language might intrigue 
more people and make them focus more on the actual video rather than 
the voiceover. ... I want [people] to watch the video, I don’t really care 
if they don’t understand, I want them to kind of see the feelings that they 
get when they watch it, just the video”. 


For Maize, the use of French could carry her message not by conveying meaning 
but by intriguing people. The audience’s inability to understand what she says 
strengthens other modes of communication. People could focus on visual and 
aural elements to negotiate meanings from the relationships of design elements. 
She wanted to awaken people’s feelings through creativity, and she had found 
written texts to be merely informational and not sufficiently aesthetic. 


4.2. Oral/aural mode 


A piece of music and a recorded voiceover represent the aural mode of Maize’s 
project. 


In her interviews, she explained that selecting music was difficult, and she had to 
change it twice. She avoided music with too fast or too slow a beat or lyrics that 
could distract her audience. She played one song throughout her project. 


Her voiceover in French presents structured sentences to mitigate the language 
barrier. In the postquestionnaire, she wrote, “I didn’t think complex ideas could 
be expressed in another language. Especially if that language was choppy and 
not fully developed. But then, I found a way to voice those complex ideas in a 
simpler manner”. Maize created short and concise sentences in L2 to deliver her 
message, and worked on her intonation to express rhythm, melody, and beat and 
emphasize the limited amount of words she used. She stated: 


“The tone of voice is crucial. I’ve always been so focused on pronouncing 
words right that I never really noticed the tone I was pronouncing them 


148 


Elyse Petit 


in. I don’t think you always need words to express something when 
your tone of voice can say everything for you”. 


Fanny put in a lot of work in the voiceover to improve her speaking skills. In her 
postquestionnaire, she explained: 


“The project helped me improve my speaking the most because I had 
to revisit my spoken portions numerous times. I feel that I spent about 
an even amount of time on making the auditory and visual components 
of the project, but I feel that I prepared the most for the auditory 
component”. 


In addition, she did not add any music to her final presentation. Instead, she 
wanted her audience to listen to the story, and “wake them up to their blessings, 
bring them to a humble state of being”. 


From the start of the project, she favored using both languages to fulfill her 
mission of becoming an inspiration for others. In her prequestionnaire, she 
explained, “I would like to have the whole project in French to improve my 
French but I am concerned about my subject becoming complex with tenses 
that I have not learned yet’. Thus, Fanny focused her project on delivering her 
message rather than on improving her language proficiency or digital skills. 
Nonetheless, in her postquestionnaire, she confessed: 


“T used the online tool that speaks text for you. I used this tool to help 
improve my speaking skills [which] was improved by the project 
because I had to repeat saying words out loud over and over until 
they sounded correct. I had to listen to the online tools that speak text 
repeatedly in order to improve my speaking accuracy”. 


Despite her main determination to share her message, Fanny decided to select a 
digital tool to work on her L2 oral skills, and as a result she learned how to use 
it on her own. 
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4.3. Visual mode 


Both participants included visuals in their projects and used personal pictures 
to avoid any copyright issues. While Maize spent a lot of time producing her 
visuals to make sure they would convey her story on their own, Fanny presented 
numerous pictures displayed in a collage form throughout her slideshow. These 
pictures were mostly used to illustrate her statements rather than to create specific 
meanings. Nonetheless, in an examination and comparison of the semiotics 
used by Fanny, some slides revealed disparities between the linguistics and the 
visuals. One slide in particular is analyzed more deeply in the second part of the 
findings section. 


Maize’s entire project employed the visual mode. During an interview, Maize 
explained, “I ama [sic] visual, so I really like the pictures, and like, the sounds or 
the subtitles kind of come second, so when I was working on it, I was try [sic] to, 
I just leave [sic] them out”. Describing herself as a visually oriented person and 
a visual learner, she used images to share her message to others. For her project, 
she shot 14 different photos and 11 videos, and she explained: “I used my phone 
to take pictures and record most of the videos, two editing websites — Fotor.com 
and Ribbet.com — to edit the pictures so they would fit in the widescreen frame 
of the film, and Windows Movie Maker to bring it all together”. In addition, she 
balanced still and moving images throughout the project by adding effects and 
transitions to maintain an aspect of fluidity. 


These two participants made meaningful choices based on their digital skills and 
language confidence to deliver their stories. This underscores their determination 
to engage and play with design elements and shows autonomy and learning 
strategies. 


The following section provides insights on how the participants arranged 
semiotics and ‘translated’ meanings in their digital stories, by using the concept 
of either transformation or transduction (Bezemer & Jewitt, 2009; Kress, 2003) 
as the design process. 
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4.4. Orchestration of semiotics 


The chains of semiosis give particular attention to (1) the concept of 
transformation, defined as the process of shifting elements within a mode, and 
(2) the concept of transduction, which is “a process where something configured 
or formed in one modality is reconfigured or reformed into a different modality” 
(Kress, 2003, p. 47). 


4.4.1. Maize’s case 


The concept of transformation was particularly relevant when comparing 
Maize’s storyboard with her final project. She provided detailed explanations 
about her use of design elements and a fully developed French narrative. Figure | 
presents a sample of Maize’s storyboard. By capturing the visuals, she produced 
an original digital story and gave people “a sense of [her] own perspective, and 
what [she] believe[s] in”. 


Figure 1. Maize’s first slide of the storyboard 


Images (general or specific, location if known): 


Design (background color, font style, special effects): 
(No title screen) 

Transition into video from black screen. 

Background image: Video of people on their computers. 
Audio (music, sound effects): 

Music: Intro — The xx (song starts) 


Narration (the actual text that you would record to accompany this slide): 


La technologie. Qu’est-que c’est la technologie? 
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Maize took multiple steps to achieve her goal. First, she selected pictures she 
liked from Google images and reproduced them. Her principal concern about 
using Google images involved the fact that the images were not hers and she 
did not know about their original purpose. Using an image that looked similar 
to what she was trying to convey was not enough. She needed to capture her 
environment and give her perspective. In her second interview, she explained, 
“T did not want random people, but people with whom | interacted and talked to 
them, and asked them to act for me to portray my experience and show my view 
within my environment”. Inspired by images found online, Maize reconfigured 
and rearranged her visuals. She staged scenes and made people act to “translate” 
her vision. Figure 2 presents one of the Google images selected by Maize on the 
left’, and the picture she recreated on the right. Her act of recontextualization and 
transformation within the same mode is incontestable. In setting the scene in her 
environment — the university — and in representing people who live around her 
in realistic outfits and attitudes, she showed ownership in the process of design. 


Figure 2. Comparative table between the storyboard and the final project 


Original Picture 


(drawn from Maize’s DST) 


Google Image 


(presented in storyboard) 


Moreover, Maize had a good sense of her storyline, and her storyboard provided 
fully structured sentences written in French. She produced short sentences and 
a redundant pattern to develop a melody and communicate her message. From 
the storyboard to the final version, the sentences did not change much and she 
only corrected lexico-grammatical mistakes based on feedback. Nonetheless, 
Maize decided to stretch out some sentences across the slides. For instance, in 


2. To the best of this researcher’s knowledge this image is copyright free. 
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Table 1, the second sentence in bold of Slide 3 from the storyboard (Nous le/ 
la véerifions a chaque minute) was stretched out to Slide 4 in the final version. 
The rationale for adding images and stretching out sentences was to provide 
connections between images and words. During an interview, Maize explained, 
“T kind of had really good pictures that I wanted definitively put in there, and 
some good lines that I wanted to put in there”. Thus, Maize had to stretch out 
sentences of the voiceover and played around with the aural and visual modes 
during the editing phase to finalize the project the way she envisioned it. 


Table 1. Comparative table of narrative script and voiceover 


Slides | Narration written in Slides | Narration transcribed 
the storyboard over from the voice 
of the digital story 
1 La technologie. Qu’ est-que 1 La technologie, qu’ est-ce que 
c’est la c’est la 
technologie? technologie? 
2 Comment est-ce que nous 2 Quand est-ce que 
Putilisons? nous l’utilisons? 
3 Nous lutilison [sic] tous les jours. | 3 Nous l’utilisons tous les jours 
Nous le vérifions chaque minute. 
4 Nous la vérifions a chaque minute 


Furthermore, Maize emphasized her overall message within two or three modes 
of communication. For instance, one of Maize’s pictures represented “vanity”, 
and the words she wanted to emphasize in her narration were the two adjectives 
“vain” and “negative”. In order to highlight both words, she drastically 
accentuated her intonation when pronouncing them. Taken with TechSmith’s 
software Camtasia, Figure 3 represents the transduction of meanings produced 
by Maize from one mode — visual — to another — oral. As shown, the sentence 
“nous devenons [we become]” is presented by the two encircled longest audio 
tracks, while the shortest ones represent the adjectives “vain [vain]” and “négatif 
[negative]. The increase of the waveforms (encircled) shows the verbal/textual 
elements expressed by Maize’s voice. In contrast, the decrease in volume (almost 
nonexistent waveforms) presents the music piece she used. She faded it in and 
out to create negative space and lengthen her sentence, a strategy that provoked 
suspense for her audience. 
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Figure 3. Screenshot of the combination of image, audio track, and text 


0:00.07:00 


Nous devenons vain Nous devenons négatif 


[We become vain We become negative] 


4.4.2. Fanny’s case 


By choosing a PowerPoint slideshow as her medium and selecting only specific 
design elements, Fanny engaged in transformation and transduction processes 
that were subtler and overlapping in her project. Her use of L1 and L2 to address 
her audience, her translation of the captions, and her image choices presented 
more complex engagement in the design process. 


Fanny decided to use written captions to illustrate her story. However, she used 
both languages unevenly based on the message she conveyed and her target 
audience. At the beginning of her project (Slides 2 and 3), she stated, “Toute ma 
vie j’ai eu la famille et les amis qui m’ont supporté dans tous mes efforts. Je n’ai 
jamais été traité [sic] différemment parce que j’avais un bras [sic] [All my life, 
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I have had family and friends who supported me in all of my endeavors. I was 
never treated differently because I had one arm]”. Whereas the translation is 
pulled from the storyboard, these two slides appeared exclusively in French in 
both modes, textual and oral, in Fanny’s final project. She did not use English 
captions. Moreover, her statement is strong and frames her story. She carefully 
selected the French definite articles /a and /es in “J'ai eu la famille et les amis”, 
excluding anyone who is not part of friends or family, and told her audience that 
she had not missed out on anything in her life: “Je n’ai jamais été traité [sic] 
différemment”. 


Yet, from Slides 4 to 6, she added English captions and used French only for 
the voiceover. To address the English audience, she started her story with a 
statement of uncertainty: “Sometimes I wondered if my parents knew that I had 
one arm” and illustrated both slides with a collage of pictures portraying herself 
with specific family members: her parents and grandparents. At this point, 
she spoke not only to an English audience, but a specific one: her family. She 
included them after presenting herself to the French audience, stating that she 
had what she needed and rejecting any compassion or judgment from them. 
Starting her story at different slides based on a specific audience underscored 
how she constructed meanings through the use of L1 and L2. 


Moreover, from the proposal to the final version, she changed the title of her 
story from “One arm wonder” to “Ma vie [my life]’, a more general title that 
addressed both the L1 and L2 audiences and encompassed anyone who would 
like to listen to her story. More importantly, she talked to her family and shared a 
more intimate story. Although Fanny’s goal in her prequestionnaire was to give 
“a chance for the audience to be enlightened to a change in perspective’, the 
findings highlighted layers of meanings developed in the arrangement of modes. 


Furthermore, while at the beginning she firmly stated to the French audience 
that she had never been treated differently (Slide 3), she confessed later 
(Slide 6) to both audiences: “when I was young, I never imagined that people 
would treat me differently... As I grew older, I experienced many people who 
thought differently”. This slide introduced a series of events in which she 
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faced people who only saw her as a disabled person. She narrated these events 
in Slides 7, 8, 9, and 10, in which she displayed images and captions in both 
French and English. Fanny did not embed a voiceover in any of these four 
slides. 


Findings revealed that Fanny had never provided Slides 7 and 8 as she turned in 
the different steps of her project. She added them in her final version. Without 
a postinterview, I could not explain the participant’s choice to add these pieces 
to her story. Since she did not receive feedback on these oral parts, I could only 
suppose that Fanny did not embed them because of her lack of confidence in her 
speaking skills. Thus, she avoided recording them on her own, and instead, put 
captions in both languages. 


Nevertheless, I believe that Fanny’s decision was deliberate and carefully 
thought out. First, she chose to present all events without a voiceover even 
though she had recorded and received feedback for Slides 9 and 10. Then, she 
provided captions in the same order, French first and English second. Finally, 
she differentiated each language with a different color, red for French and green 
for English, and used this differentiation throughout the entire project. 


Moreover, toward the end, Fanny continued to play with captions in French 
and/or English. Slides 11, 13, and 14 provide English captions, while Slide 
12 does not. Slides 15, 16, and 20 present both languages. Slides 17 and 18 
have English captions only, and Slide 19 is presented exclusively in French. 
At first glance, her choices are uneven and seem random, but an in-depth 
examination of how Fanny orchestrated elements of design revealed careful 
attention to detail and engagement in the design process. These disparities that 
emerged between and within modes point out meaningful decisions in her use 
of L1 and/or L2 based on (1) the audience(s) she targeted, (2) the message she 
conveyed and how she expressed it, and (3) the confidence she had with her 
own language proficiency. 


Finally, the transduction process is particularly relevant in Fanny’s selection 
of visual and linguistic elements. Two examples (Slides 7 and 8), in which 
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Fanny recounted two events, propose to examine how she constructed layers of 
meaning and developed agency. 


Both experiences occurred during a trip to Australia in which she participated 
with a group of students. In Slide 7, she explained that she was not allowed to 
hold a koala, while other students did receive permission. Despite a detailed 
narration, she focused the audience’s attention on one single picture to bring 
more meaning to her words (Figure 4). 


Figure 4. Representation of Fanny’s Slide 7 


Quand les gens n’avaient pas la foi en 
moi, j'ai dO me rappeler avoir la foi en 
moi-méme. Cette image a été prise 


quand j‘étais en Australie. On a permis 


2 tou! monde de tenir le koala, mais 


pas a moi. 


J'ai dG essayer difficilement de ne pas 
pleurer dans cette image. Bien que 
j'aie voulu tenir le koala et je savais 
que je pourrais, j'ai donné au gardien 
de koala ma paix parce qu'il n'a 
simplement pas compris. 


When people did not believe in me, | 
had to remember to believe in my self. 
This picture was taken when (was in 
Australia on a student ambassador trip. 
All of the students were allowed to 
hold the koala but not me. 


I had to try my hardest not to cry in 
this picture. | wanted to hold the koala 
and | knew that ! could, { gave the koala 
keeper my peace because he simply 


did not understand. 


He was only protecting the koala. That 
Il protégeait seulement le koala. Que je | understand that 


le comprenne. 


Every picture that Fanny chose throughout her DST portrayed her smiling and 
happy. Yet, for this particular moment, she informed her audience that she was 
frustrated. She was torn between the koala keeper’s intention to keep the animal 
safe and her desire to prove her ability to hold the animal safely. According to 
the way she phrased it in both languages, her decision to not argue with the koala 
keeper showed her inner strength. 


“T had to try my hardest not to cry in this picture. I wanted to hold the 
koala and I knew that I could, I gave the koala keeper my peace because 
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he simply did not understand. He was only protecting the koala. That 
I understand that”. 


In providing this afterthought, she maintained a positive attitude, avoiding 
blaming anyone who did not understand her, and preventing her audience 
from feeling uncomfortable or targeted. The adverbs ‘simply’ and ‘only’ and 
the redundancy of ‘that’ underscored self-reflection and her acceptance of the 
situation. Nonetheless, the disparity between her words “I had to try my hardest 
not to cry in this picture” and the picture displaying her smiling, encouraged the 
audience to think about the emotions she could feel as a disabled person. Through 
her words, she shared her feelings, while the visual showed the opposite. She 
smiled when posing for the photo; nevertheless, in her story, she meant to warn 
her audience about the ways people could act toward disabled people and the 
lack of understanding they could potentially demonstrate. 


In the second example, Slide 8 (Figure 5), the concept of transduction occurred 
within the translation from L1 to L2. In the slide, she was describing outdoor 
activities such as zip-lining, surfing, and scuba diving in which she participated 
with her group and the coaches’ attitude when giving safety instructions. 


Figure 5. Representation of Fanny’s Slide 8 


J'ai eu beaucoup de batailles avec moi-méme. 
Avant que les photos de moi, le surf et la 
plongée sous-marine a été prise, Les instructeurs 
hésitaient a me laisser participer et m’a 
demandé plus d'une fois « Ctes-vous sOre que 
vous pouvez le faire ? » Leurs visages 
exprimaient un certain scepticisme et de la pitié. 


Et ensuite bien sir, cette petite voix dans ma 
téte est venue disant quelque chose comme 

« Ouais ratée, va V'asseoirl Pour qui te prends- 
tu? » Mais au lieu d'écouter cette voix, je l'ai fait 
taire et j’ai continué 4 vivre ma vie, 


Before the photos of me zip lining, surfing, and 
scuba diving were taken, the instructors were 
hesitant about letting me participate. They 
asked me several times questions like, “Are you 
sure you can do it?” “Have you done anything 
like this before?” Their faces expressed 
complete skepticism, pity, and extreme doubt 


And of course, that Jittle voice in the back of my 
head came saying phrases similar to, “Yeah go sit 
down, who are you kidding?” But instead of 

listening to that yoice | silenced it and continued 


surt experience 


to live my life. 
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During the narration, she decided to use a mini-dialogue happening between 
herself and her inner voice: “cette petite voix dans ma téte est venue disant 
quelque chose comme ‘ouais ratée, va t’asseoir! Pour qui te prends-tu?’ [that 
little voice in the back of my head came saying phrases similar to, ‘Yeah go sit 
down, who are you kidding?’]”. 


This dialogue, created when working on her project, never happened in real life, 
and her intention was merely to provide an example of how she was interpreting 
the coaches’ thoughts about her disability. “They asked me several times 
questions like, “Are you sure you can do it, have you done anything like this 
before’? Their faces expressed complete skepticism, pity, and extreme doubt”. 
Her words “complete skepticism”, “pity”, and “extreme doubt” are striking, and 
by using all of them in one sentence, Fanny seemed to concentrate on the many 
feelings that she had seen in others’ perceptions. They accentuated her disability 
and diminished her humanity. For the first time, she exposed some anger and 
frustration through the use of the “little voice”. 


Moreover, the expressions that Fanny used to make her inner voice speak were 
much more provocative in French than in English. She employed a familiar 
language register by saying, “ouais ratée” [yeah, failure] that she did not use 
in English. Also, she used the form “tu” which in the French context conveys 
familiarity between two interlocutors. She also used the imperative mode and 
an exclamation mark to communicate her perception of the coaches’ command. 
Finally, the question “Pour qui te prends-tu?” is usually employed to recognize 
someone’s inappropriate behavior in a given situation, or someone who is 
exceeding their rights. 


By using these expressions in L2, Fanny demonstrated her ability to nuance 
what she would like to communicate in French and her native tongue. The 
French version seemed more bitter and heartless than the English version. Fanny 
may have deliberately softened her English version because of the broader 
or more intimate audience she anticipated to reach. Knowing that the French 
audience would only be restricted to her instructor and her classmates, who 
might not notice the differences as language learners, Fanny may have intended 
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to (1) show some language proficiency by using some familiar expressions that 
she had learned during her semester abroad; and/or (2) express deeper feelings 
with no consideration toward the audience’s (mainly my) reaction. Nonetheless, 
she developed her voice in her L2 as she made linguistic choices on how to share 
this particular event with her L2 audience, creating different feelings between 
the L1 and L2 messages. 


5. Discussion 


In this study, I did not analyze language proficiency and instead examined the 
use of the target language with equal attention to other semiotic resources. The 
study suggests that students made intentional language and semiotic choices to 
construct meanings and express opinions. 


Maize realized that sharing her point of view could be difficult in French 
despite her high proficiency. To mitigate challenges, she committed to work on 
diverse linguistic designs, including intonation, personal pronouns, and present 
tense. She chose to make short and well-structured sentences that she used as a 
pattern. Her vocabulary choices were selective and precise, which allowed her 
to sharpen her intonation and to emphasize specific words by stretching them 
out and orchestrating them with her visuals. She took the time to select music 
that supported her voiceover intonation and intertwined them. In addition, she 
shot her videos, choosing particular camera angles and movements. She stepped 
into the role of a director by interacting with people, explaining her project, 
deciding upon the setting, and directing them as actors. She used all the spatial 
and gestural design elements available to her and created new ones to design 
meanings. 


Fanny used all the digital features required by the instructions, but her story 
was primarily based upon the writing and speaking portions. She carefully 
chose her words in both languages and worked on her intonation. She used 
informal discourse in which the French expressions were provocative and 
incisive. Whereas she selected the linguistic mode as the main design element, 
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she carefully chose her pictures and thoughtfully displayed them to illustrate 
her narration, mixing slides with collage and slides with single photos as a 
focal point. She demonstrated her understanding of how the visual aspect of 
her storytelling embellished and supported her story and directed the way she 
represented herself. 


As this study suggests, creating DS allows students to combine old and 
new literacies (Sylvester & Greenidge, 2009), which impact their language 
development (Hur & Suh, 2012). Through the creation of digital stories, 
students “articulate a complete, coherent story with a beginning, middle, and 
end in the target language using multiple media and multiple modalities” 
(Castaneda, 2013, p. 57). The benefits of integrating DS in FL classrooms relate 
to language production and practices. According to Smeda, Dakich, and Sharda 
(2014), digital narratives give students the opportunity to select and concentrate 
on the language modality(ies) they want to improve. They can also focus on 
specific tasks, such as structuring complete sentences, pronouncing high-level 
vocabulary, spelling words, and practicing intonation (Hur & Suh, 2012; Kim, 
2014; Ramirez Verdugo & Alonso Belmonte, 2007; Smeda et al., 2014). Digital 
stories enhance language modalities that allow students to contextualize and 
construct their stories through the use of linguistic structures and visual and 
audio aids. As Darvin and Norton (2014) pointed out, “[w]hile some learners can 
be particularly skillful in crafting a story through words, others may be good at 
choosing images, finding the right music, matching the different elements, and 
using the digital tools” (p. 62). 


6. Conclusion and implication for FL education 


Today’s necessity to integrate new literacy approaches (Cope & Kalantzis, 2015; 
Kern, 2015, Paesani, Allen, & Dupuy, 2016) into FL curricula to promote 21st 
century skills is clear and should not be overlooked by schools, administrators, 
and teachers. However, such curricular changes cannot be realized without 
urgently addressing the needs of FL teachers through quality professional 
development opportunities. Researchers have reported that teachers’ lack of 
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preparation, awareness, and familiarity with the ways a multimodal text works 
in its interplay between modes and meaning-making constructions prevents 
them from integrating and teaching multimodal literacy in their classrooms 
(Chandler, 2017). 


Thus, through the lens of DS, this study illustrates the implementation of the 
multiliteracies pedagogy in an FL course and can serve as a potential model 
for FL teachers. DS is a tool that serves multiple teaching objectives and 
fosters numerous language modalities and learning strategies. Podkalicka and 
Campbell (2010) suggested that digital stories are more valuable to the process 
of learning than the final product itself. In developing personalized learning 
experiences (Smeda et al., 2014), DS invites students to become more aware 
of their language needs and skills and become proficient in technical aspects. In 
various studies (Robin, 2008; Smeda et al., 2014; VanderArk & Schneider, 2012, 
as mentioned in Smeda et al., 2014), teachers have reported that the ‘learning by 
doing’ approach fosters students’ self-confidence to ask questions, participate in 
discussions, and express opinions. 


Moreover, working with digital literacies increases students’ motivation (Kim, 
2014) and collaboration (Castaneda, 2013; Smeda et al., 2014), and helps them 
to remain engaged throughout the project (Sylvester & Greenidge, 2009). 
However, teachers who consider integrating digital stories into their curriculum 
must have clear goals and objectives. They need to know the reasons behind 
such a project, their expectations, and what they want to assess. They must be 
trained and prepared to support students’ needs to achieve 21st century skills. 


Future studies should take into account how a more extended period (i.e. several 
semesters) could impact students’ performance, and consider students’ level 
of language proficiency to understand how the learning process affects their 
semiotic awareness. In addition, future projects using DS could integrate into 
the curriculum (1) the editing phase to facilitate the creation process, and (2) a 
participative platform where students could post their productions, watch and 
comment on others’ stories, and be involved in “real-world processes of language 
use” (Ollivier, 2018, p. 36), reaching communities beyond the FL classrooms. 
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